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The Data Quality Campaign is a national, collaborative effort to encourage and support state 
policymakers to improve the collection, availability and use of high-quality education data 
and to implement state longitudinal data systems to improve student achievement. The 
campaign aims to provide tools and resources that will assist state development of quality 
longitudinal data systems, while also providing a national forum for reducing duplication of 
effort and promoting greater coordination and consensus among the organizations focusing 
on improving data quality, access and use. 


To help states learn from one another, DQC staff visited state education agencies which are 
engaged in different stages of development of their longitudinal data systems. In 2006, 
staff visited Florida, Utah, Virginia and Wisconsin; 2007 meetings were held with 
Massachusetts and South Carolina. The DQC will continue to highlight specific state efforts 
to develop and use longitudinal data systems, and will conduct additional case studies in 
2008 


This publication was produced by the Data Quality Campaign/National Center for 
Educational Accountability for submission under contract with the Bill & Melinda Gates 
Foundation. The report was written by Cherry Kugle, Ph.D. and Nancy Smith, Ph.D. and was 
reviewed and approved by the South Carolina Department of Education. 
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South Carolina: Building in Progress 

The South Carolina Department of Education (SDE) oversees 85 school districts, 
encompassing over 1,100 schools, in which more than 700,000 students were enrolled in 
the 2006-07 school year. 

As of the 2007 Data Quality Campaign (DQC) annual survey of state longitudinal data 
systems, South Carolina has implemented six out of ten essential elements. The only 
elements the state has not implemented include the ability to track untested students and 
the reasons they were not tested, student- level college- readiness test data, the ability to 
match student-level data between K-12 and postsecondary, and a data audit system. 

In 2005, the SDE received a three- year grant from the Institute of Education Sciences (IES) 
of the United Stated Department of Education (USDE) to develop a statewide student 
longitudinal data system (SC LDS). The goals of the grant include integrating the data that 
currently reside in separate systems in the state department, expanding the data to be 
collected, providing uniform standards and definitions for the data collection and most 
importantly, providing timely data to educators and policymakers to inform decisions and 
improve instruction for public school students in South Carolina. 


History 

State Longitudinal Data System: Building on existing networks: 

Beginning in 1995, the state began to connect all schools and public libraries to a high- 
speed network, using $10 million in funds provided by the state legislature. Rather than 
build a separate education network, an existing network that served approximately 40 state 
agencies was expanded. This approach resulted in wide-band internet access for all P-12 
education entities while enhancing the bandwidth for the other state agencies. Regular 
upgrades are made to the network to support increased data volumes over time. The SC 
Budget and Control Board, Chief Information Office, maintains this infrastructure for the 
schools and districts. It evaluates requests for additional bandwidth from the districts based 
on current usage, student population and anticipated growth, and submits reports to the 
federal government to request funds for this technology in supports of local education 
agencies (LEAs). 

Teacher Information Systems: In 2003, a Professional Certified Staff (PCS) web-based 
system replaced SC's distributed system for providing staff and teacher employment 
information to the SDE. Dating back to 1999, these data are added to the data repository 
at the end of each school year and used by multiple SDE offices. 

Student Information System: The school administration student information system (SASI) 
was purchased by the state for all school districts and implemented over a three-year period 
beginning in 2000. The SASI system operates as a client-server system with a file server at 
each of the 1,185 schools. Some districts have developed their own district- wide data 
warehouse. Beginning in the 2003-04 school year, data are extracted quarterly from each 
district and moved to the state using SC's custom SASI Web Extract and Export Tool 
(SWEET). The SDE works with districts to cleanse their data prior to moving data to the 
statewide data repository. 
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In 2004, the SDE formed the Data Registry Advisory Committee (DRAC) to register and 
publish a catalog of current and planned data collections. This effort has reduced the 
redundancy of data collections by the state. 

Interoperability and Student Identifiers (IDs): South Carolina implemented the student 
unique numbering system (SUNS), a student ID system based on the Schools 
Interoperability Framework (SIF) standards, for use beginning in the 2005-06 school year. 
The SUNS generates, assigns, stores, and locates unique, unchanged and unduplicated 
identifiers for all public school students in grades P-12 in South Carolina. The SDE uses SIF 
to help automate the transfer of unique student identification numbers from the state-level 
database into the appropriate local student information system. 


Development of the South Carolina Student Longitudinal Data System 

In November 2005 the SDE received a three-year I ES grant in the amount of $5.8 million to 
develop a statewide student longitudinal data system. High- level goals of the project 
include providing uniform rules for data entry and element level editing, and automating the 
error feedback and data resubmission processes between the districts and the SDE. 

Participants and Processes 

The current project team in the Office of Technology wrote the IES grant and meets weekly 
to review the progress of the project. Because of existing job responsibilities, none of the 
team members was devoted full-time to implementation of this grant. Therefore, a project 
manager and assistant project manager were hired at the beginning of the second year to 
ensure that progress on the grant proceeds as planned and to keep all stakeholders fully 
informed and engaged in problem-solving. 

During the first year of the grant, the SDE project staff worked with a consultant to conduct 
an inventory of all data elements collected, to expand the current data dictionary into an 
enterprise wide data dictionary, to identify the issues involved in fully automating the data 
collections, and to design a data architecture to serve as a blueprint for how the agency 
looks today and how it should change under the SC LDS. 

A Core Advisory Group of researchers with experience and perspective as end users of data 
provides guidance to the project. Researchers with experience working within school 
districts are included in this effort. This group has been influential in helping design the first 
application to be built using the LDS Data Warehouse. 

The Technical Advisory Committee, consisting of technology coordinators from 14 districts, 
is a standing committee that meets monthly to ensure that all plans and developments 
meet the technical needs of the state and districts. Committee composition reflects both 
large and small districts and the variety of staffing levels for technology across the state. 
The SC LDS development is a recurring agenda topic for this committee and provides an 
opportunity for review and input into the project plans. 

Presentations at statewide meetings attended by state and LEA staff - both technology staff 
and non-technology staff - were made throughout the first year of the grant to inform 
stakeholders from relevant groups about the SC LDS development efforts and solicit input 
from both potential users of the data and technology staff. 
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Current Status 

Out of the processes conducted during the first year of grant, a Request for Proposal (RFP) 
was developed to hire an automated State Data Manager. The state department was 
seeking a private sector partner to complete the automation of the SASI data collections at 
the district and school levels, put in place more a thorough application of business rules so 
districts can clean and transmit data on a more frequent basis, and give districts more 
automated tools to analyze and edit the data. 

A vendor was chosen in May 2007 to implement the State Data Manager (SDM). South 
Carolina's SDM is a Schools Interoperability Framework- based (SIF-based) system that will 
allow for a real time or nearly real time collection of student data, the use of business 
validation rules, and web-based feedback to district and school level users. As an end result, 
this system will aid the agency in complying with reporting requirements in an accurate and 
timely manner. Data collections and business rules will be centrally managed without any 
burden on the districts. As part of the contract, districts are licensed for the vendor 
products, which can be used for districts' horizontal SIF projects. This project will position 
districts to optionally host the business validation rules engine for district-specific business 
rules as well as populate a district- based data system. 

The majority of data are collected through the Office of Technology. The Office of Research, 
which has responsibility for administering the state and federal accountability systems, has 
developed web applications for districts to use in correcting the data to meet the standards 
of the research efforts. A goal of the SC LDS effort is to provide clean and timely data to 
the Office of Research to meet their reporting and analysis requirements, especially in the 
areas of computing graduation rate and longitudinal data analysis of student growth. 

Conditions that Expedite the Development of the Longitudinal Data System 

• Staff have the benefit of continuity, having worked together for many years with 
little turnover. 

• The SDE is partnering with the Office of Research and Statistics in another state 
agency (the Budged and Control Board) to ensure attainment of the goals of the SC 
LDS. 

• The project has hired a consultant who previously served as a deputy superintendent 
at the SDE and who also spent 12 years in the legislature as a researcher to 
advocate with state and education policymakers on behalf of the SC LDS. This Data 
Advocate, who also leads the Core Advisory Group, is meeting with teacher groups, 
principals, district technical staff, and Superintendents in school districts about the 
SC LDS. Future meetings are planned with state legislators and their staff. 
Prototypes are being developed to use during these meetings to show the reporting 
and analysis possibilities if one is an experienced data user and a set of point-and- 
click reports available for those who are not. 

• The SDE has designated a Data Champion within the SDE who is attending district 
meetings to support the efforts of the local technology coordinators and inform their 
leadership of the status of the SC LDS, with an emphasis on the importance of data 
quality. 

Efforts to Support the Development of the State Longitudinal System 

• expanding SIF capabilities horizontally within districts and requesting statewide 
participation in SI F 
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• seeking sources of grants and funds, and researching the appropriate budget 
request, to sustain the project after the grant expires 

• researching marketing strategies to inform stakeholders of the capabilities of the SC 
LDS 

• implementing electronic transcripts and an electronic student data exchange for 
grades P-12 and postsecondary records 

• conducting a mandated study on the feasibility of statewide, computer-based testing 

• collaborating with the Office of Education Quality on Teacher Validation Reports to 
increase the accuracy of data on highly qualified teachers 

• increasing understanding among the technology coordinators and assessment 
coordinators of the flow of data under the SC LDS 

The K- 12 J oint Records Exchange (J Rex) is a component of the SC LDS grant. I ts purpose is 
to allow school districts in the state to exchange student records from the statewide student 
records system that is installed in every school in the state. This will speed up student 
records processing when a student moves from one district to another within the state. The 
J Rex system will also allow the electronic exchange of student transcripts from high schools 
to institutions of higher education both within South Carolina and in other states. 


Costs of Developing, Using and Updating South Carolina's Longitudinal 
Data System 

Estimated costs to the state 

Project staff did not feel they could accurately assess the project costs to the state, stating 
"costs are always underestimated -- the disclaimer will be bigger than the estimate because 
so much of it is absorbed or resources are shifted around." However, some costs and 
resources can be identified based on information contained in the I ES grant application and 
obtained during the site visit. 

• SDE computer facilities upgrades (power, air conditioning, file server and ancillary 
equipment) - $505,000 

• I nvolvement of 11 of 47 full-time SDE technology personnel in the project 

• Support to poor and rural districts from the SDE Office of Technology and the South 
Carolina Budget and Control Board to keep local and wide-area networks operating 

• Schools Interoperability Framework (SIF) investment 

o $1,200,000 - Statewide SIF infrastructure and student identifier system 
o $27,500 - Annual district and state SI FA membership 
o $200,000 - Annual cost for general maintenance 

• Funds allocated annually to each district from the state budget for use in technology 

• State Data Manager - Approximately $1,500,000 for start-up costs, excluding annual 
maintenance 

• Data Warehouse - Approximately $1,000,000 for start-up costs, excluding annual 
maintenance 

• J Rex - Estimated $500,000 for start-up costs, excluding annual maintenance 

• One server per district for State Data Manager 

• Annual maintenance for Student Information System 
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Estimated costs to districts 

District staff interviewed also did not feel they could provide cost estimates. In general, 
districts that participate in piloting new systems for the state are willing to budget the time 
and pay for travel for staff to attend relevant meetings because it is ultimately important to 
all the districts in the state that the systems work and are useful at the local level. Until the 
SC LDS project began, districts that had the capacity were building and using their own 
systems at their own expense. 

Although districts are not directly funded to submit data to the SDE or to review data 
quality, district staff interviewed felt that diligence in this area was critical because valid 
decisions and evaluation of status and progress in student achievement cannot be made if 
the data are not accurate. For many years, the state funded a half position for a 
technology/data person in each district, but that funding was discontinued in the 1990s with 
the effect that data entry and data quality might not get the attention they deserve. 


Lessons Learned 

• "SDEs should spend a lot of time planning and thinking hard about what it 
intends to do and commit that to paper before it applies for grants." 

Accomplishments planned for the first year of the I ES grant took longer to complete 
because issues came up that the project team had not considered or could not have 
anticipated. For example, the state procurement officer the project team had been 
working with was replaced when the RFP was considered almost ready for 
publication. This caused a delay in releasing the RFP, although the ultimate product 
was considered to be stronger as a result of the additional review. 

• "Data quality is dependent upon the level of expertise at the local level, and 
understanding of the importance of this varies considerably across 
districts." For a number of years the state provided funding for a half position for a 
technology/data person in each district. Beginning in the mid-1990s this support 
was discontinued. Meetings with superintendents indicate that not all are familiar 
with how much data are provided to the state by school districts or the level of 
expertise the state is expecting the data entry clerks to have. Currently the Office of 
Technology Services employs a part-time trainer who visits districts to conduct data 
quality seminars. These seminars are aimed at upper level school and district 
administrators as well as data entry clerks, but there is concern that without more 
support for data entry at the local level data quality will suffer. 


Key Findings that Guarantee Success of SC LDS 

• Build Political Buy In Through Communication and Involvement : South Carolina's 
implementation of its state longitudinal data system has benefited from the use of an 
integrated communications/marketing plan. The plan utilizes the skills and 
experience of 1) a Data Advocate to communicate with various constituency groups 
(legislature, researchers, superintendents and so on) and a 2) a Data Champion who 
works closely with staff in districts around the state is critical to advocate for more 
attention to, understanding and support for and participation with changing 
technology systems to meet all their needs. No one should be caught unaware. 
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Informed stakeholders are more invested in the process and outcomes of the state 
data system. 

• Make Data Quality a Priority Beyond the SDE : While the SDE should facilitate quality 
data through clear data definitions, documentation, and training, good data quality 
begins at the point of data entry in the school. Previously, the state provided funding 
for staff time dedicated to data entry activities, but that is no longer the case. Short 
of reinstituting that funding, the SDE staff understand that they need to provide 
extensive training and support for data entry clerks so that they fully understand the 
importance of correct data entry. Discussions also centered on finding ways to 
communicate the critical value that dedicated data entry time and trained data entry 
staff are to superintendents so that they also will understand the role of their staff in 
terms of data quality as reported to the state and the U.S. Department of Education. 

• New Investments and Staff are Reguired : Designing, developing, and deploying an 
extensive statewide longitudinal data system cannot be accomplished by adding 
these new responsibilities to existing staff. Oversight of the many committees, 
meetings around the state, work with vendors and activities that must be 
accomplished in-house is not a part-time prospect. SDE staff realized within the first 
year that full-time project management personnel were needed, along with staff to 
participate in constituency meetings around the state. Large-scale changes to a 
technology system involve extensive personnel and time resources. It is not just 
about hardware and software purchases or upgrades. 
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